How to Evolve Irrational Preferences

Cooperation in a single play prisoner’s dilemma is irrational. We consider a simple model where this irrational behavior will evolve, via the evolution of cooperative preferences. These players cooperate rationally, given their preferences and common knowledge, but their preferences distort – fail to track their interests in – their situation. Generalizing, we show that distortions are predictable; our evolved agents’ preferences will track all but two problematic situations, and here they will take on cooperative distortions. These models introduce the concepts and tools of evolutionary game theory – reproductive fitness values, replicator dynamics and the genetic algorithm – to address the question: when should we expect agents to be rational and when should we expect them to be something else.